feat: add a health-aware ingestion load balancer by CoreyLeath-code · Pull Request #22 · CoreyLeath-code/SentinelAI

CoreyLeath-code · 2026-08-12T14:55:15Z

Summary

Implements issue #21 and retains the dated tracking entry in Dailylog.md.

Adds an NGINX gateway at the existing public ingestion port (8080) and keeps Go ingestion replicas internal.
Separates process liveness (/health) from Postgres-aware readiness (/ready).
Adds an optional, disabled-by-default replica header used solely by the CI scaling test.
Adds gateway telemetry scraping through the NGINX Prometheus exporter.

Files changed

nginx/ingestion-load-balancer.conf — least-connections upstream, health/readiness proxy policy, forwarded-request headers, internal NGINX status.
docker-compose.yml — removes direct host exposure of ingestion replicas; adds the public gateway and exporter; points the dashboard at the gateway.
ingestion-service/main.go — exposes liveness/readiness separately and optional test-only replica identity.
ingestion-service/main_test.go — verifies the replica-identity header is opt-in.
monitoring/prometheus.yml — scrapes gateway exporter metrics.
.github/workflows/ci-cd.yml — validates NGINX syntax and exercises three replicas, multi-replica routing, and one-replica failure resilience.
README.md — documents the gateway, scaling command, and probe behavior.
Dailylog.md — preserves the dated record and links enhancement: add a health-aware load-balancing layer for API workers #21.

Validation performed

Compared the branch against current main: 8 files changed, branch is 8 commits ahead and 0 behind.
Reviewed the Compose topology to confirm only the gateway publishes port 8080.
Added committed Go unit coverage for the opt-in observability contract.
Added a dedicated GitHub Actions smoke job that runs nginx -t, starts three replicas, probes /ready, checks multiple X-Instance-ID values, then stops one replica and rechecks readiness.

The connected GitHub workspace cannot execute Docker or Go locally; the new CI job is the authoritative runtime validation and this PR remains draft until it reports.

Risks

The NGINX upstream depends on Docker Compose DNS, so this configuration is scoped to Compose rather than Kubernetes or a managed load balancer.
Write requests are intentionally not retried by NGINX because the ingestion endpoint does not provide an idempotency-key contract.
No load/performance metric is claimed by this PR.

Follow-up recommendations

Run a measured load test against the gateway in the target deployment environment before selecting replica counts or timeouts.
Add authentication/rate limiting at the gateway when the ingestion API is exposed beyond a trusted network.
Promote the same readiness contract into the production orchestrator before deployment.

Summary by CodeRabbit

New Features
- Added a load-balanced ingestion gateway that distributes requests across multiple service replicas.
- Added separate liveness and database-backed readiness checks.
- Added optional instance identification in response headers.
- Added gateway metrics for monitoring and updated the dashboard to use the gateway.
Bug Fixes
- Improved resilience by routing health and readiness checks around unavailable replicas.
Documentation
- Documented the load-balanced ingestion architecture, scaling, and readiness behavior.
Tests
- Added automated coverage for gateway routing, replica failure handling, readiness, and instance headers.

coderabbitai · 2026-08-12T14:55:24Z

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 8299c7eb-710f-4a69-a89e-f95c18c359eb

📥 Commits

Reviewing files that changed from the base of the PR and between 8c97571 and b1ff20e.

📒 Files selected for processing (8)

.github/workflows/ci-cd.yml
Dailylog.md
README.md
docker-compose.yml
ingestion-service/main.go
ingestion-service/main_test.go
monitoring/prometheus.yml
nginx/ingestion-load-balancer.conf

📝 Walkthrough

Walkthrough

Changes

Ingestion load balancing

Layer / File(s)	Summary
Service health and instance identity `ingestion-service/main.go`, `ingestion-service/main_test.go`	The service separates liveness from database-backed readiness. Optional `X-Instance-ID` middleware covers service routes. Tests verify enabled and disabled behavior.
Gateway and replica deployment `nginx/ingestion-load-balancer.conf`, `docker-compose.yml`, `monitoring/prometheus.yml`	NGINX routes requests across ingestion replicas, retries health checks, and exposes status metrics. Compose uses readiness checks and routes Streamlit traffic through the gateway.
Deployment validation and documentation `.github/workflows/ci-cd.yml`, `README.md`, `Dailylog.md`	CI validates NGINX configuration, replica routing, and readiness after a replica stops. Documentation and the daily log describe the gateway architecture and current implementation status.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Streamlit
  participant NGINX
  participant IngestionReplicas
  participant Prometheus
  Streamlit->>NGINX: Send ingestion request
  NGINX->>IngestionReplicas: Route request across replicas
  IngestionReplicas-->>NGINX: Return response with instance identity
  NGINX-->>Streamlit: Return response
  Prometheus->>NGINX: Scrape gateway metrics

Possibly related issues

CoreyLeath-code/SentinelAI issue 21: The PR implements its health-aware NGINX load-balancing objectives.
CoreyLeath-code/Scalable-Event-Driven-Ride-Sharing-Platform issue 17: The PR covers the same gateway, replica routing, readiness, resilience testing, and documentation objectives.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch docs/log-load-balancer-gap

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

docs: record load-balancer gap

5b6ad23

CoreyLeath-code added 7 commits August 12, 2026 11:00

feat: add ingestion service load-balancer configuration

58a5095

test: cover ingestion load-balancer observability contract

0db114c

feat: separate ingestion liveness and readiness endpoints

10c99bb

feat: route ingestion traffic through a scalable NGINX gateway

f75fe70

feat: scrape ingestion gateway metrics

742d213

ci: verify scaled ingestion gateway behavior

7612f35

docs: document the ingestion gateway contract

b1ff20e

CoreyLeath-code changed the title ~~docs: record load-balancer implementation gap~~ feat: add a health-aware ingestion load balancer Aug 12, 2026

CoreyLeath-code marked this pull request as ready for review August 12, 2026 15:17

CoreyLeath-code merged commit 9fd96c6 into main Aug 12, 2026
16 of 17 checks passed

Navigation Menu

feat: add a health-aware ingestion load balancer - #22

feat: add a health-aware ingestion load balancer#22
CoreyLeath-code merged 8 commits into
mainfrom
docs/log-load-balancer-gap

CoreyLeath-code commented Aug 12, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Aug 12, 2026 •

edited

Loading

Review failed

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

CoreyLeath-code commented Aug 12, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Files changed

Validation performed

Risks

Follow-up recommendations

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review failed

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

CoreyLeath-code commented Aug 12, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Aug 12, 2026 •

edited

Loading